Summary

This PR replaces the old A/B-specific dnn-benchmarking workflow with an explicit ordered multi-engine path. It keeps engine selection on --engine, supports repeated engine IDs as distinct selections, and allows comma-delimited plugin paths that map positionally to the selected engines.

Risk Assessment

Medium risk. This changes CLI behavior and removes legacy A/B flags in a benchmark tool, but the new path is covered by parser, config, execution, reporting, full Python tests, and gfx90a hardware smoke validation.

Testing Summary

• Static Python compile validation covered the changed source and tests.
• Full dnn-benchmarking pytest coverage ran on Alola gfx90a with Python 3.12 and ROCm PyTorch, including GPU-marked tests.
• Focused regression coverage includes CLI parsing, config validation, ordered engine/plugin selection, same-engine/different-plugin paths, absolute per-engine plugin loading, per-engine setup failures, output shape, internal profiling plugin-path forwarding, and serialization.
• Pre-commit ran on all PR-changed files and passed.
• Alola gfx90a validation used the installed MIOpen provider from /opt/rocm/lib/hipdnn_plugins/engines and ran dnn-benchmarking smoke tests for single-engine, two-engine, and comma-delimited plugin-path flows.

Testing Checklist

• Python compile - python3 -m compileall -q projects/hipdnn/tools/dnn-benchmarking/src/dnn_benchmarking projects/hipdnn/tools/dnn-benchmarking/tests - Status: Passed
• Pre-commit on PR-changed files - pre-commit run --files <PR changed files> - Status: Passed
• Full dnn-benchmarking test suite - workspace/scripts/alola-session run -- 'cd /home/AMD/sareeder/worktrees/rocmlibs-benchmarking-alola-test/projects/hipdnn/tools/dnn-benchmarking && source /workspace/.venv/bin/activate && DNN_BENCH_WORKSPACE=/workspace python -m pytest tests' - ASICs: gfx90a - Status: Passed, 683 passed, 6 skipped, 8 xfailed
• Alola engine listing - hipdnn_list_engines --plugin-dir /opt/rocm/lib/hipdnn_plugins/engines - ASICs: gfx90a - Status: Passed, found MIOPEN_ENGINE and MIOPEN_ENGINE_DETERMINISTIC
• Alola single-engine smoke - python -m dnn_benchmarking --graph graphs/sample_conv_fwd.json --warmup 1 --iters 2 --engine 1563989756945604898 --plugin-path /opt/rocm/lib/hipdnn_plugins/engines - ASICs: gfx90a - Status: Passed
• Alola two-engine smoke - python -m dnn_benchmarking --graph graphs/sample_conv_fwd.json --warmup 1 --iters 5 --engine 1563989756945604898,-6748551569128940061 --plugin-path /opt/rocm/lib/hipdnn_plugins/engines - ASICs: gfx90a - Status: Passed
• Alola comma-delimited plugin-path smoke - python -m dnn_benchmarking --graph graphs/sample_conv_fwd.json --warmup 1 --iters 1 --engine 1563989756945604898,-6748551569128940061 --plugin-path /opt/rocm/lib/hipdnn_plugins/engines,/opt/rocm/lib/hipdnn_plugins/engines - ASICs: gfx90a - Status: Passed
• PR CI - GitHub PR checks - Status: Pending

Example comparison:

Results:
  engine                       status  plugin_path                           kernel_mean_ms  kernel_median_ms  e2e_mean_ms  e2e_median_ms
  ---------------------------  ------  ------------------------------------  --------------  ----------------  -----------  -------------
  MIOPEN_ENGINE                passed  /opt/rocm/lib/hipdnn_plugins/engines  0.028           0.027             0.069        0.041
  MIOPEN_ENGINE_DETERMINISTIC  passed  /opt/rocm/lib/hipdnn_plugins/engines  0.026           0.026             0.042        0.037

Technical Changes

• Removes legacy A/B runner/configuration code and old --AId, --BId, --APath, and --BPath flags.
• Adds comma-delimited --plugin-path parsing with validation for one shared plugin path or one path per selected engine.
• Preserves duplicate --engine IDs and models each requested run as an ordered engine selection with its own plugin path.
• Uses absolute plugin loading mode for per-engine plugin-path setup so plugin comparisons are isolated.
• Records per-engine plugin setup and handle creation failures as error rows instead of aborting the suite.
• Adds per-engine summary rows with raw kernel/E2E means and medians; percent deltas are intentionally not emitted because they are derivable from the raw columns.
• Serializes median timing data and plugin paths for JSON consumers.
• Documents hipdnn_list_engines --plugin-dir <path> in CLI help so users can discover engine IDs and names.
• Raises dnn-benchmarking's Python minimum to 3.12 and uses Python 3.12 tarfile data filters directly for tarball graph extraction.